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PROGRESS TOWARD SYSTEM INTEGRITY 


In the computer field context, the term “system integrity’ can 
be defined as the behavior of a hardware/software system that 
“does the right things and only the right things; moreover, the 
system does those things right and does them when they are 
needed to be done.” System integrity is a goal to aim at, and one 
that can never be fully realized. But it is an important goal, even 
in a business data processing environment. Imperfections in to- 
day’s computerized systems all too often lead to frustrations, an- 
noyances, or even harm. Some pioneering work in secure operat- 
ing systems may be pointing the way for achieving a higher de- 
gree of system integrity in business applications. | 


L, 1973, SRI International (formerly 
Stanford Research Institute), of Menlo Park, 
California, received a contract from the U.S. 
Department of Defense that had two main 
goals. One goal was to design a highly secure 
operating system. The second goal was to be 
able to prove that this system was secure. In 
order to permit formal (mathematical) verifi- 
cation of system properties, such as its secur- 
ity, SRI developed a rigorous methodology for 
system development. | 

The problem that SRI was tackling, as de- 
scribed by Neumann (Reference 3a), was: how 
should one design, implement, debug, operate, 
modify, and maintain a large, complex com- 


puter system that includes both the hardware 


and the operating system? Further—and this 
was a critical part of the methodology—they 
wanted to be able to formally verify that the 


system actually would do what its specifica- 
tions say that it should do, for such things as 
performance, security, reliable operation, and 
recovery from faults. 

As we have indicated in previous reports (for 
instance, May 1970, July 1977, February and 
March 1978, and January and February 1979), 
developing systems that do what they are sup- 
posed to, and only what they are supposed to, 
poses very challenging problems. It is difficult 
to determine requirements, then to develop 
specifications for the new system that meet 
those requirements, and then to build the new 
system to meet the specifications. Once the sys- 
tem is built, the problem does not end; it must 
be maintained and modified without destroying 
its desired characteristics. 

The SRI work has drawn on the results of a 
number of other projects attacking this. prob- 
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lem area, including work done at Ford Aero- 
space, the MITRE Corporation, and the Univer- 
sity of California at Los Angeles. In addition, 
similar work has been going on in other coun- 
tries. Most of this work is not classified from a 
military security standpoint. The SRI people 
have had a lot of information interchange with 
these other projects, particularly with the 
MITRE project. Even though each project has 
followed a different approach to the problem, 
the SRI people have benefitted from this inter- 
change. 

The major results of the project to date are 
as follows, as described in Reference 1: 


Mierarchical development methodology (HDM). 
This is the methodology for developing sys- 
tems that supports formal verification of sys- 
tem operation, in accordance with the specifi- 
cations. It, in turn, is made up of several com- 
ponents. 

Concepts. The methodology uses hierarchi- 
cal decomposition for attacking a complex 
problem. It also uses abstraction—isolating a 
few properties of an object that are appropri- 
ate for explaining or understanding that object, 
at the particular level of the hierarchy under 
consideration. It uses modularity which sub-di- 
vides the system into easily replacable parts 
that have well-defined external interfaces. It 
uses formal specification via a somewhat-math- 
ematical specification language. Unlike natural 
language, this specification language is unam- 
biguous—each statement has one and only one 
meaning. The method employs design verifica- 
tion, to prove that specifications are consistent 
with formal requirements. It also uses program 
verification for determining the consistency be- 
tween a program and its specifications. 

Procedures. HDM divides the development 
cycle into eight stages, starting with conceptu- 
alization and an informal statement of the 
problem, and ending with the production of 
verified code. It is being extended to cover sub- 
sequent incremental changes for improvements 
and maintenance. Note that conventional pro- 
gram testing and debugging is greatly reduced; 
the formal verification process performs the 
same function much more effectively. 

Languages. HDM uses three specification 
languages. SPECIAL is used for specifying and 
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representing modules. The operations that the 
module must perform are specified indepen- 
dently of how they will be performed in the 
implementation. Also, HSL is a language for de- 
scribing levels and hierarchies of levels. Fi- 
nally, ILPL is an intermediate level program- 
ming language, used for describing abstract 
programs. Alternatively, a modern program- 
ming language such as Ada, Modula, or Euclid, 
may be used directly. 

Tools. The people at SRI have developed 
prototype tools for several of the development 
stages. These tools include a module checker, a 
representation checker, an interface checker, 
and a hierarchy checker. Several other tools 
are either in development or have been pro- 
posed. 

HDM has been used on a number of projects 
dealing with complex systems. These include 
three projects on secure operating systems, one 
on a highly reliable flight control system, and 
one on a family of real-time operating systems. 

As an example of the use of HDM, we will 
briefly describe one of the secure operating 
system projects. 


Provably secure operating system (PSOS). At a 
session of the IEEE Computer Society Comp- 
con Spring 79 conference, E. L. Burke of the 
MITRE Corporation, gave his views on the cur- 
rent status of secure operating systems (Refer- 
ence 2). The first generation of such systems 
has been characterized by the adoption of 
sound engineering principles to the develop- 
ment of software, he said. The approach in- 
cludes top-down design, the use of design and 
implementation tools, and the use of a ‘some- 
what simple’ formal model of the security 
problem. But first generation technology had 
to limit the scope of the problems that were 
tackled, he added. For instance, non-security 
aspects—such as denial of service—were 
Skipped. Also, the formal model did not in- 
clude the hardware, allowing the possibility for 
hardware maintenance people to gain access to 
the security system. And the tools used were 
relatively independent and stand-alone, rather 
than being an integrated set of tools for design 
and implementation. 

The second generation of secure operating 
systems is just beginning to appear, said Burke. 


Some of the shortcomings of the first genera- 
tion are being overcome. And one example of 
a second generation secure operating system is 
PSOS. 

As discussed by DeLashmutt (Reference 2) 
and Feiertag and Neumann (Reference 3b), 
PSOS has been designed to be a secure operat- 
ing system that is independent of specific hard- 
ware/software boundaries and of specific hard- 
ware. However, the hardware must be able to 
support the concept of capabilities, to be dis- 
cussed shortly. The capability concept can be 
used equally well as the protection mechanism 
for user programs, application sub-systems, and 
system software. No special protection is 
needed for system software. 

Further, PSOS is designed to provide multi- 
user, multi-security-level service. As an exam- 
ple of one of the first generation shortcomings 
that it has overcome, it protects against one 
user getting into the domain of another. And it 
protects against a user being able to see resi- 
dues in memory left by previous users of the 
memory resource. PsOs has many other similar 
features. 

Feiertag and Neumann report that the de- 
sign of PSOS has been formally specified, using 
SPECIAL. The specifications define PSOS as a 
collection of about 20 modules, organized in a 
hierarchical structure. The design has been 
completed to the point where the features of 
PSOS can be compared with those of other se- 
cure operating systems. 

Psos has not yet been implemented. One 
reason is that no single, commonly-available 
computer has all of the features needed for im- 
plementing Psos efficiently, although all of the 
required features do exist in some hardware. 

As an example of a hardware support feature 
needed by PSOS, consider the concept of a ca- 
pability, as used in PSOS. In order to gain ac- 
cess to an object (such as data or another pro- 
gram) that is controlled under PSOS, a user pro- 
gram must present an appropriate capability to 
the module that is responsible for that object. 
A capability is a token (a string of bits) that in- 
cludes (1) a tag, (2) a unique identifier, and (3) 
a set of access rights, telling what operations 
the user program may perform on the object. 
The tag is critical; it distinguishes a capability 
item from, say, a data item. The hardware 
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must make the tag field inaccessible to pro- 
grams, so that capabilities cannot be forged or 
altered by user programs. 

The formal techniques used in the PSos de- 
sign both make implementation straight-for- 
ward and make the formal verification of cor- 
rect operation possible. So the pPsos design 
should lead to much more secure and reliable 
operating systems than are now commercially 
available, say the authors. 

In short, they see HDM as providing a way to 
build software that does everything its specifi- 
cations say it should do, and nothing it should 
not do. 

The problem of system integrity is pervasive 
in computer-based systems. It appears to us 
that projects such as this one at SRI may be 
charting a path that many system developers 
can eventually use. In order to build highly se- 
cure operating systems, they must find a way to 
develop software (to operate in a specific hard- 
ware environment) that does exactly what it is 
supposed to, and nothing else. In most business 
applications, the requirements will not be as 
severe. But the principles for achieving soft- 
ware integrity will still apply. 


The need for system integrity 


Definitions of the word ‘integrity’ in dictio- 
naries usually include something like “the con- 
dition of behaving justly, properly, and hon- 
estly, according to standards of good behavior” 
and give as an illustration the phrase a man of 
integrity. In the computer field, the term seems 
to be quite widely used but with several dif- 
ferent connotations. “Data integrity’ generally 
means the “the condition of being whole, com- 
plete, accurate, and timely,” for instance. And 
the term has been applied also to both pro- 
grams and systems. 

Our use of the term ‘system integrity’ will 
mean what we indicated at the beginning of 
the report—namely, the behavior of a hard- 
ware/software system that does the right 
things and only the right things; further, it does 
these things right and does them when they are 
needed to be done. We should note, however, 
that not only is this term not widely used in 
this manner but also that there is some contro- 
versy about it. The controversy centers on 
whether this is a valid use of ‘integrity.’ But we 


feel that a system can exhibit integrity in much 
the same way that a person can, so that “system 
integrity’ is a valid concept. 

Neumann, in Reference 3a, uses the term 
‘system defensiveness’ to denote a part of what 
we include in ‘system integrity—namely, the 
non-existence of inappropriate behavior, as 
much as possible. He says that the components 
of defensiveness include security, reliability, 
availability, recoverability, and auditability. 

System integrity is particularly important for 
systems with severe operating constraints. 
Some of these include air traffic control, the 
control of nuclear power generation, vote 
counting, and certain law enforcement systems, 
where errors of operation can have very seri- 
ous consequences. Similarly, the security of 
classified military information is very impor- 
tant; when such information is stored in com- 
puter systems, the need for secure hardware/ 
software is evident. 

The business data processing environment 
has a need for system integrity, but usually not 
as extreme as the cases just cited. The need for 
data security is becoming recognized, and will 
become even more significant as countries pass 
privacy laws that control the transfer and dis- 
closure of information about persons. And, of 
course, as organizations begin to store sensitive 
information such as company plans, trade se- 
crets, etc. on computers, and control the trans- 
fer of their funds by computers, security will 
be essential. 

But even in more mundane applications, 
such as billing, system integrity is needed. And 
the history of computerized billing systems 
certainly indicates that their integrity has not 
been as high as might be desired. Systems that 
continue to send erroneous bills (and adding 
more penalties each time), or dunning for zero 
balances, have undermined some of the pub- 
lic’s respect for computers. 


One might then ask: What is so difficult 
about getting systems with an adequate level 
of integrity? 


Some of the problems 


In our previous issues on this general sub- 
ject, listed earlier in this report, we have given 
some idea of why it is so difficult to achieve 
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system integrity. We will briefly review some 
of the causes. 

The life cycle of a system has the following 
main components: (1) a new system begins by 
considering the mission of the activity that the 
system is supposed to serve—what really should 
the system do, and why; (2) then comes the re- 
quirements for the new system; ideally, the re- 
quirements define what the system must do, in 
support of the mission; (3) the specifications 
for the new system follow; these are (formal) 
statements of what the new system must do, 
based on the requirements; (4) next is the de- 
sign of the new system, which should be in har- 
mony with the specifications; (5) the imple- 
mentation of the new system follows; it should 
be in harmony with the design; (6) next is the 
test of the new system, to assure the develop- 
ers and users that it does what (the mission? 
the requirements? the specifications?) say it 
should; (7) then there is the maintenance of the 
new system, to remove detected errors; and fi- 
nally there is (8) the enhancements and changes 
made to the new system, during its life. 

Studies have shown (as we reported in the 
previous reports) that many of the problems 
with new systems can be traced to errors in the 
requirements statements. The errors include in- 
correct requirements statements, missing or in- 
complete statements, unclear or ambiguous 
statements, and inconsistent or incompatible 
statements. 

So a part of the problem of obtaining system 
integrity has to do with the inadequacy of 
methods for determining and stating require- 
ments. We discussed some improved methods 
for doing this in our January and February is- 
sues of this year. 

Requirements errors often are not detected 
until the new system has been built and is be- 
ing tested. It would be much easier and less 
costly to correct them had they been found by 
the specification or design stages. 

Another part of the problem comes from 
moving from one stage to the next. In going 
from requirements to specifications, some addi- 
tional errors may be injected. The same is true 
in going from specifications to design, and so 
on through the whole life cycle. 

In short, almost regardless of how talented 
or experienced the development staff is, errors 


will creep in. These are errors both of omission 
and commission. Methods are needed to flush 
out the errors as soon as possible. 

Also, the problem of shortcomings in system 
integrity can apply to just about all computer- 
ized systems, from quite small to large. One 
relatively small interactive system that we 
have studied, for instance, required about five 
man-months to design, construct, and debug. 
The user has been quite happy with the result- 
ing system, which has been in daily use for 
over two years. But, in actuality, the system 
has had some integrity faults. There are a num- 
ber of functions that the user requested which 
the system does not do. And in a few instances, 
the system does things that it is not supposed 
to do. Some of the shortcomings were due to 
an inadequate statement of requirements, ac- 
cording to the user. But the types just men- 
tioned—system not doing what it should and 
doing things it should not—probably were 
caused by the use of an inadequate methodol- 
ogy. 

The methodology used, in this case, con- 
sisted of the user supplying the implementer 
with a fair amount of informal documentation 
on requirements, followed by a greater amount 
of verbal communication. No formal require- 
ments statements were developed, nor were 
specifications developed. Design and program- 
ming drew on the requirements documents and 
verbal communications. 

This approach is not untypical today, we 
suspect. Some larger users are attempting to 
put more formality in the development 
process, as we have discussed in other reports. 
But many of the medium size and even larger 
size organizations are still using an approach 
similar to the one just described. And as the 
use of micro-computers spreads through 
smaller organizations, the same type of thing 
will happen—again and again. 

At the other end of the size and complexity 
spectrum, Neumann (Reference 3a) discusses 
some types of system design and construction 
practices that have been found to cause secur- 
ity flaws in operating systems. Some examples 
are: a poor choice of security system bounda- 
ries, thus allowing users to get at security-criti- 
cal functions; users allowed to use absolute in- 
put-output addresses; operating systems that 
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assign two different names to the same object, 
or the same name to two different objects; and 
a lack of validation of critical conditions and 
operands, such as outside-of-bounds input val- 
ues. 


These and similar problems can be traced to 
deficiencies in the development methodologies 
used, says Neumann. Such deficiencies include 
the lack of formally stated requirements, for- 
mally stated specifications, formal proof of cor- 
respondence between specifications and_ re- 
quirements, and so on. So even though operat- 
ing systems have usually been developed by 
very capable people, the same general types of 
shortcomings mentioned above for a small sys- 
tem are found to occur.in many large operat- 
ing systems. 

Neumann compares two of today’s operating 
systems, MULTICS and UNIX, for security fea- 
tures. MULTICS was designed (in the mid- 
1960s) to provide strong security features; 
UNIX was designed in the early 1970s for use in 
a benign environment and makes little pretense 
at being secure. (Neumann said he ignored the 
conventional commercial operating systems in 
his evaluation because they are, for the most 
part, intrinsically insecure.) He concludes that 
MULTICS is relatively secure, but points out 
some areas of possible weakness. UNIX, on the 
other hand, is not secure—apart from a basic 
set of read/write protect bits. This is not a 
criticism of UNIX, he says, because it was not 
designed to be secure. Rather, he simply wants 
to show how insecure an operating system can 
be in the absence of a real concern for security 
during its development. 


Neumann made the point to us that security 
is only one aspect of what we have called sys- 
tem integrity. Some aspects of integrity will be 
imporant in every system; security may or may 


not be an important aspect in a particular case. 


In general, it appears that any desired aspect 
of system integrity—performance, reliability, 
security, availability, etc.—-must be actively 
sought by the developers. It will not happen as 
a matter of course or by assuming that “since 
we are using only top-notch people on this 
project, we won't have to worry about system 
integrity.” System integrity is a generic prob- 
lem that applies to all systems, from small to 


large, and from relatively simple to horribly 
complex. 


Elements of system integrity 


About two years ago, a committee of the 
American Federation of Information Process- 
ing Societies (AFIPS) organized a by-invitation- 
only workshop to explore the subject of system 
integrity. As the ultimate objective, AFIPS ho- 
ped to have a ‘best practices’ manual on sys- 
tem integrity developed to their specifications, 
along the lines of the AFIPS Security Manual. 
The workshop was asked to explore the ques- 
tion, consider whether such a manual was do- 
able, indicate the type of coverage that might 
be expected, and suggest how an author search 
might be conducted. 

While additional work was done after the 
workshop was held, for a number of reasons 
the project has been put into a ‘hold’ condi- 
tion. Hopefully, the project will be re-acti- 
vated; we think such a manual is both impor- 
tant and needed. 

For discussing the elements of system integ- 
rity, we will draw upon the (unpublished) re- 
port of that workshop. We should mention at 
the outset that the workshop participants saw 
two primary audiences for the proposed man- 
ual: application system designers, and manag- 
ers (both of data processing and of user depart- 
ments). Also, included among the participants 
were several people from the business data 
processing field, so common DP applications, 
such as billing, were within the subject area 
that was discussed. 

The workshop participants viewed system 
integrity as consisting of a number of factors or 
elements. A system should be available, ready 
to serve users when the users want to use it. 
The system should be appropriate, in that it 
does the right things, and bounded, in that it 
does only what it is supposed to. And the sys- 
tem should be correct—what it does, it does 
right. The system should be predictable by al- 
ways doing things the same way, and should be 
timely by doing things at the right time and 
delivering results when needed. The system 
must be maintainable and not lose integrity in 
the process of being fixed or enhanced. And it 
should be auditable, so that auditors can verify 
the integrity of the system. The workshop 
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pointed out that these factors seemed to cover 
such concepts as reliability, security, recovery, 
change control, and so on. Further, the factors 
are inter-related, so that trade-offs often will 
have to be made among them. 

These factors perhaps indicate the ‘ideal’ in 
system integrity. But the workshop participants 
felt that levels of system integrity may have to 
be considered, from a practical viewpoint. 


Levels of integrity 


Intuition says that the concept of levels of 
system integrity is a valid one; system integrity 
is not an all-or-nothing matter. For example, 
an on-line word processing system serving, say, 
ten user stations does not need the same degree 
of availability, correctness, predictability, and 
timeliness as does a control system for a space 
probe. 

So levels of system integrity are determined 
not only by which of the elements listed above 
must be emphasized but also by the degree of 
emphasis given each one. For a control system 
with high reliability requirements, not only 
should availability be emphasized, it probably 
should be considered a critical factor, and the 
cost of providing it probably should not be 
given the same weight. 

The workshop participants felt that the lev- 
els of system integrity might be partially deter- 
mined by the tightness of coupling between 
the system and its end users. For instance, 
some batch systems can be down for hours 
without end users being aware of the problem. 
But if an on-line sales order entry system goes 
down, terminal operators are aware of it im- 
mediately. 

Another aspect that would seem to influence 
the levels of system integrity is important so- 
cial values. Computers are being put into sys- 
tems where faulty performance is intolerable. 
Examples are air traffic control and the control 
of nuclear power plants. But even within the 
business data processing environment, social 
values must be considered. As indicated earlier 
in this report, the designers of a good many of 
the computerized billing systems did not ade- 
quately consider social values—the frustrations, 
the annoyances, and even the actual harm that 
their systems have caused—when they designed 
the systems. 


In fact, end user satisfaction or dissatisfac- 
tion with a computerized system is really a so- 
cial value. In this sense, the designers of all 
computer-based systems should be concerned 
with social values, such as the threshhold 
where user procedures cease to be acceptable 
and become frustrating. System software often 
provides examples of disregard for social val- 
ues—the frustration users get from supplying 
input to some operating systems is exceeded 
only by their frustration trying to decipher out- 
put error messages. 

Higher levels of system integrity cost larger 
amounts of money to accomplish. So system 
developers must determine the appropriate 
level for a new system. And this does not ap- 
pear to be easy to do, especially since humans 
are involved in all systems and the human ele- 
ment makes system integrity less predictable 
and controllable. 


End user involvement. In order for system in- 
tegrity to be maintained, there are some things 
that end users must perform. They must know 
system limitations and not try to force the sys- 
tem beyond its limits. (Hopefully, of course, 
system designers should check all user interac- 
tions for outside-of-bounds values and protect 
against any such.) But even more important, 
they must understand their role in the overall 
system and its relationship to system integrity. 

During system development, it is essential 
that the developers obtain end user participa- 
tion for determining what the system should do 
and what it should not do. For this, they need 
contact with the real users, not intermediaries. 
Obtaining this type and amount of end user 
support is not always easy. 

In short, end users have an important role to 
play in achieving an appropriate level of sys- 
tem integrity, both during the design and con- 
struction of the system as well as during its op- 
erating life. 


Life cycle considerations. Application systems 
change and evolve through time; they do not 
remain static. We have been told of studies 
that report about 80% of development staff 
costs, on the average, are spent on mainte- 
nance and enhancements of application sys- 
tems. The original development represents 
only 20% of the total. Further, other studies 
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have pointed out that the original structure of 
a system tends to be destroyed by changes and 
enhancements. 

What this means is that system integrity can- 
not be considered just during system develop- 
ment and then put aside. The desired level of 
system integrity must be maintained during the 
whole life cycle of that system. 

It appears to us that the concept of levels of 
system integrity is valid. Designers must select 
the appropriate level for any given system. The 
appropriate level is a function of the tightness 
of coupling between the system and its end 
users, as well as of the social values held by the 
users. The effects of the behavior of both users 
and maintainers on system integrity must be 
considered. And in determining the appropri- 
ate level of integrity, the cost of both provid- 
ing and maintaining that level over the life cy- 
cle of the system must be determined. These 
are not easy things to accomplish. 

A project, such as the one contemplated by 
the AFIPS committee, might be able to locate 
practical methods for accomplishing these 
things. It would be very useful, we believe, to 
have such methods collected and published. 

But computer-based systems are being built, 
and in rapidly increasing numbers. Their de- 
signers typically would like to meet the (prob- 
ably poorly stated) integrity goals. In the face 
of all of the above factors, how might system 
developers achieve a desired level of system in- 
tegrity? A part of the answer is: use formal 
methods. 


Use of formalism 


Formal methods become more and more im- 
portant as the severity of constraints on the 
system increases. At the least, formal methods 
require written documentation, formal review 
techniques, and standard approaches. The stan- 
dard approaches, in turn, involve standard pro- 
ject phases, standard documentation for each 
phase, and standard procedures to make sure 
that all detected errors have been properly cor- 
rected. 

As the discussion of HDM has indicated, for- 
mal methods move into the realm of mathe- 
matics as the severity of the constraints passes 
some threshhold. That is, the design of a highly 
secure operating system (probably) cannot be 


accomplished without the precision of mathe- 
matical tools. 

Many system designers in the business world 
are not too proficient in mathematical meth- 
ods, we suspect. So mathematical formalism, 
of the type used in HDM, say, will have to be 
‘translated’ into a form more appropriate for 
such designers. That has yet to be done. Be- 
cause the need exists and the rewards for ac- 
complishment will be substantial, we believe 
that it will be done. In the meantime, those 
system developers who can handle the mathe- 
matical procedures of HDM (or other such 
methodologies) may well decide to begin using 
them. 

Before outlining some of the components of 
formalism, it would be well to mention a cou- 
ple of problems related to it. One problem is 
the apparent delay formal methods seem to 
cause—the ‘huge’ increase of time spent in the 
requirements, specifications, and design stages 
of a project. It is not unusual for some people 
to finally say, ““Let’s cut out all this foolishness 
and begin writing some code.” (Neumann 
points out that implementation and mainte- 
nance time savings more than offset this delay.) 

Another problem area is that formalism can 
become bureaucratic, especially if the people 
do not understand the basic concepts but only 
know the procedures to be used. Then the con- 
cern is more with form than with substance. 

It seems to us that both of these problems, 
and other similar ones that might arise with 
the use of formal methods, can be adequately 
dealt with by an interested and concerned man- 
agement. But this means that management 
must understand what the formal methods are 
attempting to do, and why. This is a problem 
area in itself. 

With this background, let us now look at 
some of the components of formal methods, as 
identified in the workshop. 

Needs assessment. Formal methods already 
exist for determining what the new system 
must do. For instance, we discussed SADT and 
IA in our January issue of this year. Both of 
these are graphical languages, and have the ad- 
vantage of communicating well with end users 
without the use of jargon or mathematics. Hav- 
ing determined requirements and documented 
them with such a language, it might then be 
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desirable to state the requirements in a mathe- 
matical-type language, for later proving corre- 
spondence between requirements and specifi- 
cations. 

Another need exists at this level—that is, a 
method is needed by which the appropriate 
level of system integrity can be determined. To 
our knowledge, this question has not been ad- 
dressed by any of the methodologies we have 
encountered. The problem is either ignored (in 
most cases), or else maximum integrity is 
sought. 

Formal specifications. We gather, from our 
discussions in the field, that many of the short- 
comings that have occurred in computer-based 
systems could have been avoided through the 
use of formal specifications. These specifica- 
tions state, from the designer’s point of view, 
what the new system must perform. They rep- 
resent the designer’s understanding of the 
user's requirements. 

HpM provides a specification language, SPE- 
CIAL. By using this language for stating both 
the requirements and the specifications, it is 
possible (claim the developers) to prove the 
correspondence between the requirements and 
the specifications. It seems to us that this facil- 
ity provides a big step forward for system in- 
tegrity. 

Modes of failure analysis. In addition to tel- 
ling what the system should do, the specifica- 
tions should also fully describe what it should 
not do. In other words, the system analysts and 
system designers should make a thorough anal- 
ysis of the ways that the system might (a) not 
do what it is supposed to and (b) do what it is 
not supposed to. The specifications should then 
specify actions to guard against these possibili- 
ties. 

Formal design documentation. Correspon- 
dence between specifications and design must 
be established, in much the same manner as 
between requirements and specifications. For- 
mal design documentation will help to accom- 
plish this. 

Hardware/software selection. In order to 
meet the appropriate level of system integrity, 
the hardware, operating system, and purchased 
software must be considered. A manual, such 
as the one contemplated by AFIPS, might pro- 
vide a checklist of things to look for, plus 


some “good practices’ that might be followed 
when selecting hardware and software. 

In many (most?) instances, of course, appli- 
cation system developers will have little or no 
say about the hardware or operating system 
that is to be used. But, at least, if the deficien- 
cies of the hardware/software are known, this 
would reduce the chance of unrealistic expec- 
tations about the overall system integrity. It 
would also indicate what hardware difficulties 
the software should attempt to overcome. 

Test plans and quality assurance. A method 
for verifying that the completed system agrees 
with its specifications is essential. The most 
widely used method for this, by far, is testing— 
that is, provide a variety of types of input and 
see that the proper outputs are produced. 

During the design and construction stages, 
this testing can be simulated by means of de- 
sign reviews and code inspections. The devel- 
opers can ‘walk through’ a variety of input 
types, for a small group of reviewers, and indi- 
cate what outputs will be produced. This is a 
slow process, and can be used for only the ma- 
jor input types, we gather. 

The formal verification methods used in 
HDM, however, apparently bypass much of this 
conventional testing. By logically demonstrat- 
ing that a program does exactly what it is sup- 
posed to, and nothing else, the need for testing 
that program is greatly reduced. Formal veri- 
fication methods are very new, require an abil- 
ity with mathematics, and are rather difficult to 
apply. Where the need exists for a high level 
of system integrity, however, the use of formal 
verification may be essential. 

Formal training. Since human behavior can 
have such an effect on system integrity, the 
users of the system should be trained in what 
to do and what not to do. They should under- 
stand what the system limits are and should be 
encouraged not to try to force the system to 
operate beyond those limits. 

The training program is needed not only at 
the outset, when the new system is being in- 
stalled, but also must be repeated as new peo- 
ple join the organization. 

Formal turnover. As the level of needed sys- 
tem integrity increases, so does the need for a 
formal procedure to move the system from de- 
velopment to production status. There would 
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be a number of things that the production peo- 
ple would want to check before accepting the 
system. Further, this formal turnover should 
insure that no changes are made to the system 
that have not gone through formal change con- 
trol. 

Failure analysis. Achieving and maintaining 
a high level of system integrity requires that all 
system failures be recorded, analyzed, and cor- 
rected. Formal procedures are required for 
this, as well as for making sure that the correc- 
tions have been made, made correctly, and in- 
stalled. 

Auditing, Financial audit procedures tend to 
be formal in nature, in that well-defined proce- 
dures are used. However, audits for security, 
privacy, and integrity features are not yet as 
well defined; a definition of their audit proce- 
dures is needed. 

It also should be mentioned that the integ- 
rity features should be auditable, to make sure 
that they have not been compromised. Ideally, 
this means that there should be some way of 
verifying that the object programs being used 
are faithful translations of the source programs 
and that the source programs correspond with 
their formally verified designs, and so on back 
to requirements. 

Change control. There is probably no aspect 
of system integrity that is more important than 
change control. As indicated earlier, some 80% 
of staff time goes into maintaining and enhanc- 
ing existing systems, over the life of these sys- 
tems, and only 20% goes into the original de- 
velopment. When changes are made, errors 
can creep in (or can be implanted), which com- 
promise system integrity. 

So formal change control procedures should 
be used, for reviewing, approving, making, and 
verifying all system changes. 

The change control procedures should apply 
to hardware (CPUs, memory, peripherals, com- 
munication equipment, etc.), software (operat- 
ing system, utilities, application programs, and 
so on), programming languages used, as well as 
the procedures used for developing, maintain- 
ing, and operating the systems. 

It should be recognized that this need for 
change may be caused by changes in the oper- 
ating environment. These include new applica- 
tions, growth or decline in business volume, 


mergers or acquisitions, restructuring of an or- 
ganization, change in company goals, markets, 
or operating policies, and so on. 

Here, then, are at least some of the types of 
formal methods that could be used for achiev- 
ing a desired level of system integrity. In a 
sense, one can feel discouraged by all that is 
needed in order to achieve system integrity. 
However, we suspect that the various compo- 
nents will be packaged and sold commercially 
not too many years hence, with a good number 
of automated support tools included. 

Even from our brief summary, it should be 
apparent that not all of the needed formal 
methods yet exist. But a good start has been 
made toward developing these formal meth- 
ods. To illustrate this, let us now look more 
closely at HDM. It may be the forerunner of the 
‘high integrity’ development methodologies we 
anticipate. 


HDM 


As mentioned above, HDM was conceived for 
developing large, complex software systems 
where a very high degree of system integrity is 
required. An example of such a software sys- 
tem is a multi-user, multi-security-level operat- 
ing system. But the methodology can also be 
applied for the development of smaller and/or 
less stringent systems. 

HDM is a sophisticated methodology that is 
based on scientific principles—principles which 
include the concept of data abstraction and the 
mathematical basis of programming. 

Our discussion here draws upon a report by 
Levitt, Neumann, and Robinson (Reference 4) 
on the use of HDM for developing software. 
The language used and the example that is de- 
scribed in that report are appropriate for the 
intended audience—namely, designers and im- 
plementers of large, complex software systems 
such as operating systems. People who are in- 
volved in the development of business applica- 
tion software may find the report a bit difficult 
to translate into familiar terms. 

Following are some of the principles upon 
which HDM is based. 

Hierarchy of levels. HDM uses hierarchy for 
handling complexity. It calls for the decompo- 
sition of the design of a new system into a se- 
ries of levels. The top level is the user inter- 
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face. The bottom level (generally) may be the 
hardware upon which the system will run or 
the programming language in which it will be 
written. 

Abstract machines. The methodology views 
the new system as made up of one or more ‘ab- 
stract machines’ at each level of the hierarchy. 
The term ‘abstract’ is used to indicate that only 
those details that are appropriate for a particu- 
lar level will be considered; all others will be 
hidden at that level. The term ‘machine’ im- 
plies that, in theory, a mechanism could be 
conceived that performs at that level. For ex- 
ample, one might conceive of a hardware/soft- 
ware complex that had to be told only to “run 
payroll,’ and it would be able to determine ev- 
erything that had to be done. We are not refer- 
ring to the case where the whole payroll sys- 
tem has been programmed in the conventional 
manner; we are postulating the case where the 
hardware/software figures out what has to be 
done. The lower level abstract machines spec- 
ify the ‘what has to be done.’ 

At the top of the hierarchy, the abstract ma- 
chine deals with what is supplied by the user 
and, in turn, what must be delivered to the 
user. At the bottom level, the abstract ma- 
chines deal with how those user services will 
be provided by the computer. 

Modules. Each abstract machine consists of 
one or more modules, where a module is a pro- 
gramming unit that is independently imple- 
mented. That is, the internal details of a mod- 
ule are hidden from everything outside of the 
module. A module can call on a lower level 
module for service. 

Stages. HDM consists of eight stages, divided 
into three main categories. The first three 
stages make up design, the next two make up 
representation, and the final three are imple- 
mentation. Generally, the stages are performed 
in order, in the sense that all design decisions 
are made before moving on to the representa- 
tion and implementation decisions. However, 
backtracking can (and probably will) occur. 

The authors point out that HDM does not re- 
quire top-down development. For instance, 
when considering a particular abstract ma- 
chine, the designer might well also be consid- 
ering the lower level abstract machines that 
will be needed to implement that machine. 
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Also, it is quite possible, they say, to do top- 
down design and bottom-up implementation. 

Before discussing the HDM stages in more de- 
tail, several comments are in order. As men- 
tioned earlier in this report, today’s ‘conven- 
tional’ development methodologies typically 
use stages that are named: problem definition 
(or determining requirements), system specifi- 
cations, system design, program design, coding, 
test, installation, and maintenance. The HDM 
stages are similar, but they do differ from these 
in important ways, as we will discuss. Testing 
is still used, but the need for it is reduced, and 
HDM is being extended into the maintenance 
area. Secondly, HDM uses the SPECIAL language 
for the design and representation stages, and 
ILPL (intermediate level programming lan- 
guage), or another programming language, for 
the implementation stages. Thirdly, correctness 
verification is distributed across stages 1, 4, 
and 7 of HDM, and is not just limited to a sin- 
gle stage of the project. And finally, the prob- 
lem definition step (requirements determina- 
tion) is not covered by the authors, other than 
the re-statement of the requirements in the for- 
mal language SPECIAL. 


The design stages. The first three stages of 
HDM are concerned with design. 


Conceptualization. In this stage, the intent 
of the new system is described in natural lan- 
guage. The services of the top-level module 
(the user interface) and the components of the 
lowest level module(s) are indicated. This con- 
ceptualization indicates that the design of the 
new system is beginning to take shape in the 
designer's mind. However, later stages may 
Show the need to re-think this conceptualiza- 
tion. 


Decomposition of the top and bottom levels 
into modules. It is possible that the top-level 
abstract machine, and/or the bottom level 
one(s), may only require one module each. In 
complex systems, multiple modules are more 
likely. As indicated earlier, each module is de- 
fined to be a ‘stand-alone’ programming unit, 
in the sense that its internal details are hidden 
from everything outside the module. 


Intermediate level definition comes next. All 
of the levels between the top and the bottom 
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levels are defined and decomposed into mod- 
ules. 

Stages 1 to 3 are particularly important, say 
the authors, for three main reasons. For one 
thing, the results of these stages define what 
‘correctness’ means for the system—what the 
system must do. It is against this definition that 
the later stages will be verified for correctness. 
Also, the documentation of these stages pro- 
vides an early, understandable explanation of 
the system—and a good opportunity for detect- 
ing design flaws. Finally, if these stages are 
done properly (that is, if they result in a good 
choice of abstract machines and modules), then 
the following stages will probably turn out to 
be simple, even for complex systems, according 
to the authors. 


The representation stages. Representation 
consists of two stages which probably repre- 
sent the ‘technical heart’ of the methodology. 

Module specification is reasonably complex. 
We will be able to treat only a few of the 
highlights. 

The HDM specifictions have been designed to 
have three significant properties, say the au- 
thors. They can be read and understood by a 
diverse group of people, to aid in inspections 
and critiques. They are machine processable, 
so as to provide for automatic checking for 
syntax and other types of errors. And, finally, 
they need not be compilable; rather, they de- 
scribe the functional behavior of a collection 
of programs that implement a module. 

Types of expressions. There are five main 
types of expressions through which the func- 
tions of a module are described. One, of 
course, is arithmetic and another is conditional 
(‘if...then...’). Then there is a simple boolean 
type of expression, the values of which are ei- 
ther true or false. Next, a relational expression 
may be constructed from two simple boolean 
expressions, such that if one is true, then the 
other must be true. The final type of expression 
is “quantified,” to express properties relating to 
a large number of values. An example of a 
quantified expression is, “For all values of x 
such that P(x) is true, then Q(X) is also true.” 

Functions. The functions that a module per- 
forms are of three basic types: (a) one type re- 
turns a value to a requesting function, but does 
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not change the state of anything within the 
module, (b) another type changes the state of 
something within the module but does not re- 
turn a value, and (c) in the third type, both 
things occur—a value is returned and a state is 
changed. 

An example of the first type, where a value 
is returned but no state is changed, is the an- 
swer to the question, “What is the next availa- 
ble buffer location?’ Nothing within the buffer 
is changed, and the answer can be the location 
or can be ‘none,’ if the buffer is full. 

The other two types of functions involve a 
change in the state of something within the 
module. Since state-changing is critical, this is 
where the “modes of failure’ analysis plays a 
key role. All exceptions should be identified 
and the consequent actions to be taken defined. 
In HDM, each exception is named and is given 
a boolean definition of the condition under 
which it is true. 

A state-changing operation (of a module) 
thus either performs the requested state chang- 
ing and returns the ‘normal’ value, if appropri- 
ate, or else it returns one of the exceptions. 
Each type of exception is checked, in turn, to 
see if it is true. If an exception is true, the op- 
eration returns the value of that exception 
(such as ‘buffer full’). If none of the exceptions 
are true, then the normal value is returned, 
such as filling the next buffer location and pro- 
viding the pointer for that location. 

It bears repeating that SRI developed the 
SPECIAL language for specifying modules in 
these terms. 

Data representation. Somewhat in parallel 
with the development of the specifictions for 
the modules of an abstract machine, the data 
structures that will be used by that abstract 
machine must be defined. These definitions are 
made in terms of the data structure of the next 
lower level machine, say the authors. 

As we see it, an example might be the fol- 
lowing. Consider a payroll application where 
one level of abstract machine might be titled 
‘compute pay. This abstract machine might 
consist of two modules—‘compute gross pay’ 
and ‘compute net pay.’ At the next lower level, 
‘compute gross pay’ would be considered to be 
an abstract machine that might consist of four 
modules for computing regular pay, overtime 
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pay, sick pay, and vacation pay. At the higher 
level abstract machine (‘compute pay’), the 
data might be specified only as ‘gross pay data 
segment, without giving attention to what 
makes up that segment. But at the lower level 
(‘compute gross pay’), it becomes clear to the 
designer that data will be needed for regular 
hours worked, overtime hours, sick time, and 
vacation time. 

These, then—module specification and data 
representation—are the stages that make up 
the representation phase of the development 
project. The next stages are implementation. 


Implementation stages. It is in the implemen- 
tation stages that the use of SPECIAL ceases. In- 
stead, a form of pseudo-coding is first used, af- 
ter which the programming language in which 
the modules are to be written is used. 

Abstract implementation (of the operations 
of each module within an abstract machine) is 
performed by writing an abstract program for 
each module within the abstract machine. 
Each program ‘calls’ on lower level modules, 
as appropriate. 

SRI has developed ILPL (intermediate level 
programming language) for writing this level 
of code. While other languages could be used 
for this purpose, ILPL has the advantage of be- 
ing compatible with the syntax, type checking, 
and other parts of HDM, say the authors. 

In use, we gather, the programmer first 
writes an ‘informal’ set of code for a module, 
using natural language sentences—an informal 
pseudo-code. Statement types include ‘re- 
trieve, ‘modify,’ and ‘if,’ and describe what the 
module is to do. When this has been done to 
the programmer's satisfaction, it is converted 
to ILPL, a formal pseudo-code. 

The ILPL code is amenable to a variety of 
types of automated checking, for detecting er- 
rors. The reason for using ILPL is, we gather, 
that it is very easy to write from the informal 
pseudo-code, since the two involve the same 
level of detail. But once in ILPL, the code can 
be automatically checked, for flushing out er- 
rors, before the regular coding begins. 

Coding. When the tools are available, the 
ILPL code can be automatically converted into 
regular code of the programming language 
that is being used. In the absence of such trans- 
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lation tools, this code conversion can be done 
manually. Since the module design has been es- 
tablished and most of the errors removed, this 
coding tends to proceed very rapidly, we are 
told. 


Verification. As mentioned earlier, formal 
verification under HDM takes place during the 
conceptualization, module specification, and 
coding stages. At SRI, they have performed 
such verification at all three stages. For in- 
stance, they have performed the design proof, 
are engaged in doing module code proofs for 
selected modules, and have a module verfica- 
tion system in operation, for the KSOS operat- 
ing system. They also are starting on proofs of 
Pascal code for a high reliability airplane con- 
trol system, for which some design proofs have 
been done. So verification methods are at least 
available for experimental approaches to ‘real- 
world’ systems. 


This, then, is a brief overview of HDM. It has 
been designed to support the creation of hard- 
ware/software systems that exhibit very high 
integrity. As we have attempted to point out in 
this report, higher levels of system integrity are 
needed than are found in many of today’s busi- 
ness data processing applications. System de- 
velopment methodologies of the future are al- 
most sure to stress this need and their ways of 
meeting it. HDM provides a glimpse of how fu- 
ture methodologies will approach high integ- 
rity. 

But a word of caution is in order. HDM is 
still under development; it is not yet ready for 
everyday use. When it is ready for the market- 


place, chances are that learning to use it will 
be a real challenge for many system analysts 
and designers. Of course, it may spawn less de- 
manding methodologies that can be used when 
less-than-extreme levels of system integrity ap- 


ply. 

If you hear complaints that “new computer 
systems still do not do what they are supposed 
to,” HDM illustrates how difficult it really is to 
achieve good system integrity. 
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